Image-based system to observe and document eye responses having a reflective protractor for measurement of stimulus position

ABSTRACT

The image-based system is used to observe and document eye responses which can be used to implement the procedures defined in the Standardized Field Sobriety Test as well as the Drug Recognition Expert Program. These eye responses include pupil movement, as well as pupil size and responses, for either one or both eyes simultaneously. The system includes a headset against which the subject places their face. The faceplate has an opening through which illumination devices transmit beams of infrared light to illuminate the subject&#39;s eyes. A reflective protractor, having a scale imprinted thereon, is mounted on the headset above the eye opening and facing the test administrator to provide an image of the stimulus target that is used by the test administrator. Two imaging devices are mounted on the headset opposite the eye opening to generate images of the subject&#39;s illuminated eyes. The generated images are transmitted to an image recording device for storage and an image display device for display to a test administrator.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of U.S. patent applicationSer. No. 11/398,009 titled “Image-Based System To Observe And DocumentEye Responses” filed on 5 Apr. 2006.

FIELD OF THE INVENTION

This invention relates to determining the level of impairment of asubject and, in particular, to an apparatus that can be used toimplement the measurement of a subject's eye gaze tracking and themonitoring of the subject's eye behavior.

BACKGROUND OF THE INVENTION

It is a problem to determine whether a subject is impaired due to theuse of alcohol or drugs, or whether the subject is suffering from sleepdeprivation and/or a medical condition that could cause impairment. Thisimpairment determination can be reliably accomplished by the measurementof eye gaze tracking and the monitoring of eye behavior (e.g., pupilsize, blink rate, gaze, etc.) to determine a subject's alertness andattention.

For the detection of impairment, law enforcement officers in the fieldpresently ask the subject to direct their gaze at a target which followsa predefined pattern defined by the Standardized Field Sobriety Testand/or Drug Recognition Expert protocols as established by the NationalHighway Traffic Safety Administration (NHTSA). By tracking the eyemovements and measuring the pupil size and its responses, it is possibleto determine a subject's impairment status. For instance, a subject whohas been smoking marijuana demonstrates a very slow pupil reaction toabrupt changes in lighting conditions, where the pupil regains a dilatedstate even under bright light, sometimes called ‘rebound dilation’.While under the influence of heroin, the subject's pupil remains fullyconstricted, even in the dark i.e., pinpoint pupil. If a subject hasbeen drinking alcohol, their eye movements become ‘jerky’ on movementsto the right or left, i.e., ‘lack of smooth pursuit’. Upon lateral gaze,they can also exhibit a constant ‘jerk’, called horizontal gazenystagmus (HGN). Therefore, the tracking of changes in the pupil'scentroid location and/or size and reaction to light changes can be usedto indicate impairment.

With all of these metrics, the accurate observation, tracking, andmeasurement of the subject's pupil centroid location and area iscritical to the determination of impairment. It is also important insome applications to observe and detect eye blinking in order to computethe eye blinking rate to determine drowsiness and to ensure that pupilmeasurements are not computed at those time instances when a blinkoccurs, since the pupil is either partially visible or not visible atall when a blink occurs.

Presently, all of these measurements are typically performed manuallywith the accuracy and reliability of these measurements being subjectiveand possibly tainted by the ambient conditions under which the test isadministered. Ambient conditions include the inability to maintain thesubject's gaze in a fixed direction, erratic lighting conditions, andthe subjective nature of the test administrator's administration of thetest. Alternatively, standard methods of video taping of the eyeresponses for the Standardized Field Sobriety Test have been used, suchas having one camera mounted on a tripod or handheld and aimed at thesubject, but these applications can also be tainted by the ambientconditions under which the test is administered, further includingcamera aiming and maintaining a steady video image when the video camerais handheld.

BRIEF SUMMARY OF THE INVENTION

The above-described problems are solved and a technical advance achievedby the present image-based system to observe and document eye responseshaving a reflective protractor for measurement of stimulus position(termed “image-based system to observe and document eye responses”herein) which can be used to implement the procedures defined in theStandardized Field Sobriety Test as well as the Drug Recognition ProgramThese eye responses include pupil movement as well as pupil responses toillumination and target tracking for either eye or both eyessimultaneously.

The image-based system to observe and document eye responses includes aheadset that has a faceplate against which the subject places theirface. The faceplate has a single large opening through whichillumination devices transmit beams of infrared light to illuminate thesubject's eyes. Another embodiment of the system can be two openings inwhich the illumination transmits to each eye. In addition, a reflectiveprotractor, having a scale imprinted thereon, is mounted on the headsetabove the eye opening and facing the test administrator to provide animage of the stimulus target that is used by the test administrator. Inthis embodiment, two imaging devices are mounted on the headset oppositethe eye opening(s) to generate images of both the subject's illuminatedeyes and the reflected image of the stimulus target in the reflectiveprotractor. It is conceivable that other embodiments could use a singleor multiple imaging devices. The generated images are transmitted to animage recording device for storage and to an image display device fordisplay to a test administrator. The test administrator is able therebyto view the subject's eye(s) and the present position of the stimulustarget while positioning the stimulus target (finger, pen tip, etc.) forthe subject to view through the eye opening(s) of the faceplate. Thesubject's eye responses, in tracking the stimulus target, as well as theposition of the stimulus target as shown by the scale on the reflectiveprotractor as it is moved by the test administrator, are recorded on animage recording device along with audio annotations from the testadministrator.

An optional hood can be used to cover the headset from the back of thefaceplate to behind the image recording devices. The hood makes theinterior dark by the elimination of any ambient light, thus allowingvisualization of the subject's dark eye(s) on the image display device.This displays a true pupil size without external illumination.

This system allows for playback of the eye responses and associatedaudio for review and training purposes. The use of infrared illuminationof the subject's eyes further enhances the observation, as well as thedocumentation and review of the eye movement and pupil responses,especially in those individuals with dark pigmented irises. In addition,a data processing system optionally may be used to perform an analysisof the image generated by the imaging devices to quickly and accuratelylocalize the pupil's boundaries, area, and center coordinates inclose-up video imagery of the subject's eyes as captured with theimaging devices.

The use of the headset increases the accuracy and reliability of thesemeasurements and allows for testing even in low light or totally darkenvironments, which is sometimes the case in law enforcementapplications. Thus, this system can be used both in a classroomenvironment for training purposes and in the field by law enforcementofficers for testing individuals who are stopped for suspicion ofDriving Under the Influence (DU) or Driving While Impaired (DWI).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a block diagram of the present image-based system toobserve and document eye responses;

FIGS. 2-4 illustrate rear, top, and side views, respectively, of theheadset of the present image-based system to observe and document eyeresponses;

FIGS. 5 and 6 illustrate side and front views, respectively, of theheadset of the present image-based system to observe and document eyeresponses as used by a subject;

FIGS. 7 and 8 illustrate side and front views of the headset of thepresent image-based system to observe and document eye responses,including a reflective protractor for measurement of stimulus position,and to observe and document eye responses as used by a subject; and

FIG. 9 illustrates the headset of the present image-based system toobserve and document eye responses as used by a subject, equipped with ahood to create a dark environment for administering the test to asubject.

DETAILED DESCRIPTION OF THE INVENTION

In determining whether a subject is impaired due to the use of alcoholor drugs, or whether the subject is suffering from sleep deprivation orother medical impairing condition, the measurement of eye gaze trackingand the monitoring of eye behavior (e.g., blink rate, gaze, etc.) can beused to determine a subject's alertness and attention.

The accuracy and reliability of these measurements are subjective andcan be tainted by the ambient conditions under which the test isadministered. Ambient conditions include the inability to maintain thesubject's gaze in a fixed direction, erratic lighting conditions, andthe subjective nature of the test administrator's administration of thetest.

System Characteristics

In order to minimize the impact of ambient conditions, the presentimage-based system to observe and document eye responses includes aheadset that has a faceplate against which the subject places theirface. The headset increases the accuracy and reliability of thesemeasurements and reduces any inaccuracy occasioned by the ambientconditions under which the test is administered.

The faceplate has a single or two eye openings through whichillumination devices transmit beams of infrared light to illuminate thesubject's eyes. Two imaging devices (more or less in other embodiments)are mounted on the headset opposite the eye opening(s) to generate animage or images of the subject's illuminated eyes. The generated imagesare transmitted to an image recording device for storage and an imagedisplay device for display to a test administrator. The testadministrator is able thereby to view the subject's eye(s) whileproviding the stimulus target (finger, pen tip, etc.) for the subject toview through the eye openings of the faceplate. The subject's eyeresponses are recorded on an image recording device along with audioannotation from the test administrator. This system allows for playbackof the eye responses and associated audio for review and trainingpurposes. The use of infrared illumination further enhances theobservation, as well as the documentation and review of the eye movementand pupil responses, especially in those individuals with dark pigmentedirises.

System Block Diagram

FIG. 1 illustrates a block diagram of the present image-based system toobserve and document eye responses 100 (with additional details shown inFIGS. 2-6), where a headset 101 is used to administer the test to thesubject. Headset 101 includes illumination devices 111-114 and imagingdevices 121, 122 that are attached to a frame 131, which includes afaceplate 141 to position the subject's face with respect toillumination devices 111-114 and imaging devices 121, 122. A source ofelectrical power, such as batteries, is provided to power illuminationdevices 111-114 and imaging devices 121, 122, with the batteries locatedin the grip portion 151 of headset 101. The imaging devices 121, 122 areconnected to an image recording device 102, such as a video recorder,DVD recorder, computer, or other such apparatus for capturing andstoring image data. The interconnection between headset 101 and imagerecording device 102 can be wired or wireless, using a cable 171 orwireless communication protocol, respectively. An image display device103 is also provided to present a real time representation to a testadministrator of the image generated by imaging devices 121, 122, andimage display device 103 typically is connected to image recordingdevice 102 (and/or imaging devices) via a wired or wireless connection.

In addition, a data processing system 104 may optionally be used toperform an analysis of the image generated by imaging devices 121, 122to quickly and accurately localize the pupil's boundaries, area, andcenter coordinates in close-up video imagery of the subject's eye(s) ascaptured with imaging devices 121, 122. Data processing system 104 ofthe image-based system to observe and document eye responses 100 canalso use a region growing algorithm or other image processing techniqueto extract and track the centroid location and area of the subject'spupil under difficult conditions, such as blinking, eyelid droop, headmovement, eye gaze at the extreme corners of the eyes, presence of heavyeye makeup, and where light specularities occur in or near the pupilregion. The use of a data processing system 104 eliminates anysubjectivity on the part of the test administrator and reduces thepossibility of errors due to the degree of precision in the measurementof the subject's eye gaze tracking and the monitoring of the subject'seye behavior.

Headset Implementation

FIGS. 2-4 illustrate rear, top, and side views, respectively, of headset101 of the present image-based system to observe and document eyeresponses 100; and FIGS. 5 and 6 illustrate side and front views,respectively, of headset 101 of the present image-based system toobserve and document eye responses 100 as used by a subject 105.

Headset 101 is the base user interface structure of the presentimage-based system to observe and document eye responses 100. Headset101 is based on a frame 131 to which the various elements are attached.A faceplate 141 is attached to frame 131, shaped to substantiallyconform to a subject's face, and having two eye openings 142, 143oriented opposite a subject's eyes when the subject places their faceagainst faceplate 141. A chin rest (not shown) optionally can also beprovided to assist in the alignment of the subject's face with eyeopenings 142, 143.

Attached to frame 131 are two imaging devices 121, 122, such as imagecameras of the Charge Coupled Device type (or other functionalequivalent), one that provides an image of the right eye and the otherof the left eye. The image that is generated is typically captured at arate of 30 frames/sec, at a frame size of 320×240, and at a gray scaleresolution of 8 bits. The resultant image provides adequate data for theproper determination of eye gaze tracking and the monitoring of eyebehavior. The orientation of imaging devices 121, 122 can be adjusted toplace the eyes in the center of the image that is displayed on imagedisplay device 103. Image display device 103 can be of the quad-type,meaning that each eye is displayed in one of four image quadrants on thescreen of image display device 103—typically next to each other on theupper horizontal or lower horizontal aspect ratio to give an image as ifthe eyes were seen as normal, adjacent to each other.

Faceplate 141 contains illumination devices 111-114, such as infraredLight Emitting Diodes, for the illumination of the subject's eyes. Twolight emitting diodes can be used in or adjacent to each eye opening132, 133 to ensure proper illumination of the subject's eyes. Theseinfrared Light Emitting Diodes produce invisible illumination. Oneunique attnbute of the infrared illumination is the visual observationalenhancement of the CCD image of the eye for dark-iris individuals.

A separate wireless microphone system 161 allows for the verbalannotation of remarks either from the subject or the test administratoronto image recording device 102. It is contemplated that microphone 161can be built into headset 101. It is also contemplated that headset 101is wireless, meaning that the image data as well as the audio componentwould be transmitted from headset 101 to image recording device 102 in awireless mode; no connection from a hard-wired cable would be necessary.

Reflective Protractor

FIGS. 7 and 8 illustrate side and front views of headset 101 of thepresent image-based system to observe and document eye responses 100,equipped with a reflective protractor 701 for measurement of stimulusposition, and to observe and document eye responses of subject 105.Reflective protractor 701 is mounted on headset 101 above the eyeopening and facing the test administrator to provide an image 702 of thestimulus target that is used by the test administrator. Reflectiveprotractor 701 has a scale 703 imprinted thereon to enable the testadministrator to gauge the position of stimulus target 702 via thereflected image of stimulus target 702. The test administrator is ablethereby to view the subject's eye(s) and the present position of thestimulus target while positioning the stimulus target (finger, pen tip,etc.) for subject 105 to view through the eye opening(s) of faceplate141. The subject's eye responses in tracking the stimulus target, aswell as the position of the stimulus target as shown by scale 703 onreflective protractor 701, as it is moved by the test administrator arerecorded on an image recording device 102 along with audio annotationsfrom the test administrator.

FIG. 9 illustrates headset 101 of the present image-based system toobserve and document eye responses 100 as used by a subject 105,equipped with a hood to create a dark environment for administering thetest to a subject. Optional hood 900 can be used to cover headset 101from the back of faceplate 141 to behind imaging devices 121, 122.Optional hood 900 makes the interior dark by the elimination of anyambient light, thus allowing visualization of the subject's dark eye(s)on image display device 103. This displays a true pupil size withoutexternal illumination. Since the image-based system to observe anddocument eye responses 100 uses its near-infrared illumination to lightthe eye within this totally dark chamber, and the imaging devices 121,122 are sensitive to the near-infrared illumination, this allowsvisulization of the dark eye(s) on image recording device 102. This way,a true pupil size without external illumination is determined. This testis a standard part of the Drug Recognition Expert protocol as developedby the National Highway Traffic Safety Administration and administeredby the International Association of Chiefs of Police.

Additional Functionality

An added benefit provided by the present image-based system to observeand document eye responses is that test administrators, including LawEnforcement Officers who are being trained and certified in the skillsof observing the eye signs associated with the Standardized FieldSobriety Test as well as the Drug Recognition Expert Program programs,have had little opportunity to review images of eye signs of impairedsubjects during their training sessions. The ability to review actualimage data from impaired subjects, gathered under real world fieldconditions, enhances the training of these officers by allowing them toreview their observations and comments with their instructors. By havingthese responses displayed and recorded with an enhanced image systemmakes this task that much easier and meaningful.

A further benefit is the ability to preserve the real time image datathat is collected in the field for presentation and review at a latertime. The optional data processing system can provide a precise analysisof this data, both in the field and at a later time, to perform ananalysis of the image generated by the imaging devices to quickly andaccurately localize the pupil's boundaries, area, and center coordinatesin close-up video imagery of the subject's eyes as captured with theimaging devices.

SUMMARY

The image-based system to observe and document eye responses includes aheadset against which the subject places their face. The faceplate hastwo eye openings through which illumination devices transmit beams ofinfrared light to illuminate the subject's eyes. Two imaging devices aremounted on the headset opposite the eye openings to generate images ofthe subject's illuminated eyes. The generated images are transmitted toan image recording device for storage and an image display device fordisplay to a test administrator.

1. An image-based system to observe and document eye responses,comprising: a frame; a faceplate attached to said frame, shaped tosubstantially conform to a subject's face and having at least oneopening oriented opposite a subject's eyes when said subject placestheir face against said face plate; at least one illumination deviceattached to said frame and aligned with said eye opening to illuminate asubject's eyes when said subject places their face against saidfaceplate; at least one imaging device attached to said frame andaligned with said eye openings to generate an image of a subject's eyeswhen said subject places their face against said faceplate; and areflective protractor attached to said frame and aligned with said eyeopening facing said at least one imaging device to display an image ofthe position of a stimulus target used by a test administrator.
 2. Theimage-based system to observe and document eye responses of claim 1wherein said reflective protractor comprises: a reflective surfaceextending substantially horizontally along an edge of said eye opening;and a scale imprinted on said reflective surface for enablingmeasurement of a position of said image of said stimulus target.
 3. Theimage-based system to observe and document eye responses of claim 1further comprising: an image recording device connected to said at leastone imaging device for recording images generated by said at least oneimaging device.
 4. The image-based system to observe and document eyeresponses of claim 3 wherein said image recording device comprises: adigital data recording device for storing image data from the class ofdigital recording devices including: video tape recorder, DVD recorder,computer system, and data memory device.
 5. The image-based system toobserve and document eye responses of claim 1 further comprising: animage display device connected to at least one of said at least oneimaging device and said image recording device for displaying said imageof a subject's eyes when said subject places their face against saidfaceplate.
 6. The image-based system to observe and document eyeresponses of claim 1 further comprising: a power source for providingpower to said at least one illumination device and said at least oneimaging device.
 7. The image-based system to observe and document eyeresponses of claim 1 further comprising: a handle attached to said framefor enabling a user to position said faceplate against said subject'sface.
 8. The image-based system to observe and document eye responses ofclaim 7 wherein said handle includes a compartment for housing a powersource for providing power to said at least one illumination device andsaid at least one imaging device.
 9. The image-based system to observeand document eye responses of claim 1 wherein said at least oneillumination device comprises: infrared source for illuminating saidsubject's eyes with light in the infrared spectrum.
 10. The image-basedsystem to observe and document eye responses of claim 1 furthercomprising: a data processing system, connected to at least one of saidat least one imaging device and said image recording device, forprocessing said image of a subject's eyes when said subject places theirface against said faceplate to measure said subject's eye gaze trackingand eye behavior in response to tracking a stimulus target.
 11. Theimage-based system to observe and document eye responses of claim 1further comprising: a hood attached to said frame for enclosing saidheadset from said at least one imaging device to said faceplate toreduce the presence of ambient light within a volume enclosed by saidhood.
 12. An image-based system to observe and document eye responses,comprising: a frame; a faceplate attached to said frame, shaped tosubstantially conform to a subject's face and having at least oneopening oriented opposite a subject's eyes when said subject placestheir face against said face plate; at least one infrared light emittingdiode illumination device attached to said frame and aligned with saidat least one eye opening to illuminate a subject's eyes when saidsubject places their face against said faceplate; a reflectiveprotractor attached to said frame and aligned with said eye openingfacing away from said subject to display an image of the position of astimulus target used by a test administrator; at least one cameraattached to said frame and aligned with said at least one eye opening togenerate an image of a subject's eyes when said subject places theirface against said faceplate and said image of the position of a stimulustarget used by a test administrator, and an image display deviceconnected to said camera for displaying said image of a subject's eyeswhen said subject places their face against said faceplate and saidimage of the position of a stimulus target used by a test administrator.13. The image-based system to observe and document eye responses ofclaim 12 wherein said reflective protractor comprises: a reflectivesurface extending substantially horizontally along an edge of said eyeopening; and a scale imprinted on said reflective surface for enablingmeasurement of a position of said image of said stimulus target.
 14. Theimage-based system to observe and document eye responses of claim 12further comprising: an image recording device connected to said at leastone imaging device for recording images generated by said at least oneimaging device.
 15. The image-based system to observe and document eyeresponses of claim 12 further comprising: a power source for providingpower to said at least one illumination device and said at least oneimaging device.
 16. The image-based system to observe and document eyeresponses of claim 12 further comprising: a handle attached to saidframe for enabling a user to position said faceplate against saidsubject's face.
 17. The image-based system to observe and document eyeresponses of claim 16 wherein said handle includes a compartment forhousing a power source for providing power to said at least oneillumination device and said at least one imaging device.
 18. Theimage-based system to observe and document eye responses of claim 12further comprising: a chin rest attached to said frame to align saidsubject's face with said at least one eye opening.
 19. The image-basedsystem to observe and document eye responses of claim 12 furthercomprising: a hood attached to said frame for enclosing said headsetfrom said at least one camera to said faceplate to reduce the presenceof ambient light within a volume enclosed by said hood.